Skip to content

feat(spp_api_v2_simulation): add simulation REST API#72

Open
jeremi wants to merge 5 commits into19.0from
feat/api-v2-simulation
Open

feat(spp_api_v2_simulation): add simulation REST API#72
jeremi wants to merge 5 commits into19.0from
feat/api-v2-simulation

Conversation

@jeremi
Copy link
Member

@jeremi jeremi commented Mar 5, 2026

Summary

New module providing REST API for simulation scenario management:

  • CRUD endpoints for simulation scenarios, runs, and comparisons
  • Scenario execution endpoint
  • Run results with metrics
  • Population aggregation with demographic breakdowns
  • Available group-by dimensions endpoint
  • Privacy protection via unified aggregation service (k-anonymity)

Dependencies: spp_api_v2, spp_simulation, spp_aggregation

Test plan

  • Run ./scripts/test_single_module.sh spp_api_v2_simulation
  • Verify simulation CRUD endpoints work
  • Verify aggregation endpoint returns correct breakdowns

@gemini-code-assist
Copy link

Summary of Changes

Hello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request introduces a new module that significantly expands the OpenSPP API V2 capabilities by providing a dedicated RESTful interface for managing and interacting with simulation scenarios. This enhancement allows external systems to programmatically create, update, delete, and execute simulation scenarios, retrieve detailed run results, compare different simulation outcomes, and perform population aggregations with demographic breakdowns, all while ensuring privacy protection through k-anonymity.

Highlights

  • New Module Introduction: Introduced a new module, 'spp_api_v2_simulation', to provide a dedicated REST API for managing simulation scenarios and aggregation within OpenSPP.
  • Comprehensive API Endpoints: Added a full suite of FastAPI endpoints for CRUD operations on simulation scenarios, runs, and comparisons, alongside functionality for scenario execution, retrieving detailed run results, and performing population aggregations with demographic breakdowns.
  • API Scope and Security Enhancements: Extended the API client scope model to include 'simulation' and 'aggregation' as new resource types, with 'execute' and 'convert' actions, enabling granular access control for the new API functionalities.
  • Pydantic Schema Integration: Implemented Pydantic schemas for all API requests and responses, ensuring robust data validation, clear documentation, and consistent data structures across the new simulation and aggregation APIs.
  • Service Layer Implementation: Developed dedicated service layers ('AggregationApiService' and 'SimulationApiService') to abstract the business logic, facilitating interaction between the FastAPI endpoints and the underlying Odoo models for simulation and aggregation.
  • Extensive Test Coverage: Included a comprehensive test suite covering API endpoint logic, schema validation, and service functionality for scenarios, runs, comparisons, aggregation, and API scope registration, ensuring reliability and correctness.
Changelog
  • spp_api_v2_simulation/init.py
    • Initialized the new module by importing models, routers, schemas, and services.
  • spp_api_v2_simulation/manifest.py
    • Defined the new OpenSPP Simulation API module, specifying its dependencies, maintainers, and providing a detailed description of its endpoints and design principles.
  • spp_api_v2_simulation/models/init.py
    • Initialized the models sub-module, importing 'api_client_scope' and 'fastapi_endpoint'.
  • spp_api_v2_simulation/models/api_client_scope.py
    • Extended the 'spp.api.client.scope' model to include 'simulation' and 'aggregation' as new resource types and 'execute' and 'convert' as new actions, enabling fine-grained access control for the new APIs.
  • spp_api_v2_simulation/models/fastapi_endpoint.py
    • Extended the 'fastapi.endpoint' model to dynamically add new API routers for aggregation, comparison, run, scenario, and simulation to the API V2 application.
  • spp_api_v2_simulation/pyproject.toml
    • Added build system configuration for the new module.
  • spp_api_v2_simulation/routers/init.py
    • Initialized the routers sub-module, importing and exposing the new API routers.
  • spp_api_v2_simulation/routers/aggregation.py
    • Implemented FastAPI endpoints for computing population aggregations and listing available demographic dimensions, including authentication and error handling.
  • spp_api_v2_simulation/routers/comparison.py
    • Implemented FastAPI endpoints for creating and retrieving simulation run comparisons, including data conversion and validation logic.
  • spp_api_v2_simulation/routers/run.py
    • Implemented FastAPI endpoints for listing and retrieving detailed information about simulation runs, including data conversion and filtering capabilities.
  • spp_api_v2_simulation/routers/scenario.py
    • Implemented FastAPI endpoints for CRUD operations on simulation scenarios, executing simulations, marking scenarios as ready, and converting scenarios to programs, with comprehensive validation and error handling.
  • spp_api_v2_simulation/routers/simulation.py
    • Implemented FastAPI endpoints for listing simulation scenario templates.
  • spp_api_v2_simulation/schemas/init.py
    • Initialized the schemas sub-module, importing and exposing all Pydantic schemas for the new APIs.
  • spp_api_v2_simulation/schemas/aggregation.py
    • Defined Pydantic schemas for aggregation API requests and responses, including scope definition, aggregation results, and dimension information.
  • spp_api_v2_simulation/schemas/comparison.py
    • Defined Pydantic schemas for simulation comparison API requests and responses, including comparison creation, run data, and overlap data.
  • spp_api_v2_simulation/schemas/run.py
    • Defined Pydantic schemas for simulation run API requests and responses, including run summaries, detailed run data, and various metric breakdowns.
  • spp_api_v2_simulation/schemas/scenario.py
    • Defined Pydantic schemas for simulation scenario API requests and responses, including scenario creation, update, and conversion to program.
  • spp_api_v2_simulation/schemas/simulation.py
    • Defined Pydantic schemas for simulation API, including entitlement rules, scenario templates, and detailed run and comparison responses.
  • spp_api_v2_simulation/security/ir.model.access.csv
    • Added security access rules for the new models.
  • spp_api_v2_simulation/services/init.py
    • Initialized the services sub-module, importing 'aggregation_api_service' and 'simulation_api_service'.
  • spp_api_v2_simulation/services/aggregation_api_service.py
    • Implemented a service to adapt API requests for aggregation to the underlying 'spp.aggregation.service', handling scope translation and dimension listing.
  • spp_api_v2_simulation/services/simulation_api_service.py
    • Implemented a service to manage simulation scenario CRUD, execution, and comparison, acting as an adapter between the API layer and Odoo simulation models.
  • spp_api_v2_simulation/tests/init.py
    • Initialized the tests sub-module, importing various test files for the new APIs and services.
  • spp_api_v2_simulation/tests/common.py
    • Provided common test fixtures and setup for simulation API tests, including API client, registrants, and scenarios.
  • spp_api_v2_simulation/tests/test_aggregation_service.py
    • Added tests for the 'AggregationApiService', covering basic aggregation, statistics, group-by functionality, and dimension listing.
  • spp_api_v2_simulation/tests/test_comparison_api.py
    • Added tests for the simulation comparison API, covering schema validation, conversion logic, and various comparison scenarios.
  • spp_api_v2_simulation/tests/test_convert_to_program_api.py
    • Added tests for the convert-to-program API endpoint, covering schema validation, option building, and integration with the simulation service.
  • spp_api_v2_simulation/tests/test_run_api.py
    • Added tests for the simulation run API, covering schema validation, data conversion, and handling of detailed run metrics and failed runs.
  • spp_api_v2_simulation/tests/test_scenario_api.py
    • Added tests for the simulation scenario API, covering schema validation, CRUD operations, and handling of entitlement rules and program references.
  • spp_api_v2_simulation/tests/test_scope_registration.py
    • Added tests to verify proper registration and functionality of simulation and aggregation API scopes.
Activity
  • A new module, 'spp_api_v2_simulation', was introduced, indicating a significant feature addition to the OpenSPP API V2.
  • The author 'jeremi' is listed as a maintainer in the module's manifest, suggesting their primary role in developing this new functionality.
  • Comprehensive test files have been added across various components (API endpoints, services, schemas, scope registration), demonstrating a focus on code quality and reliability for the new features.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

Copy link

@github-advanced-security github-advanced-security bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Semgrep OSS found more than 20 potential problems in the proposed changes. Check the Files changed tab for more details.

@codecov
Copy link

codecov bot commented Mar 5, 2026

Codecov Report

❌ Patch coverage is 69.41650% with 304 lines in your changes missing coverage. Please review.
✅ Project coverage is 58.18%. Comparing base (69e0e5a) to head (7570668).

Files with missing lines Patch % Lines
spp_api_v2_simulation/routers/scenario.py 25.11% 167 Missing ⚠️
spp_api_v2_simulation/routers/comparison.py 45.61% 31 Missing ⚠️
spp_api_v2_simulation/routers/run.py 58.66% 31 Missing ⚠️
spp_api_v2_simulation/routers/aggregation.py 40.00% 21 Missing ⚠️
...i_v2_simulation/services/simulation_api_service.py 85.41% 21 Missing ⚠️
..._v2_simulation/services/aggregation_api_service.py 72.00% 14 Missing ⚠️
spp_api_v2_simulation/models/fastapi_endpoint.py 43.75% 9 Missing ⚠️
spp_api_v2_simulation/routers/simulation.py 57.14% 9 Missing ⚠️
spp_api_v2_simulation/__manifest__.py 0.00% 1 Missing ⚠️
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             19.0      #72      +/-   ##
==========================================
+ Coverage   55.79%   58.18%   +2.38%     
==========================================
  Files         162      196      +34     
  Lines        9291    11210    +1919     
==========================================
+ Hits         5184     6522    +1338     
- Misses       4107     4688     +581     
Flag Coverage Δ
spp_api_v2_simulation 69.38% <69.38%> (?)
spp_base_common 90.26% <ø> (ø)
spp_programs 45.43% <ø> (ø)
spp_security 66.66% <ø> (ø)
spp_simulation 70.08% <100.00%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
spp_api_v2_simulation/__init__.py 100.00% <100.00%> (ø)
spp_api_v2_simulation/models/__init__.py 100.00% <100.00%> (ø)
spp_api_v2_simulation/models/api_client_scope.py 100.00% <100.00%> (ø)
spp_api_v2_simulation/routers/__init__.py 100.00% <100.00%> (ø)
spp_api_v2_simulation/schemas/__init__.py 100.00% <100.00%> (ø)
spp_api_v2_simulation/schemas/aggregation.py 100.00% <100.00%> (ø)
spp_api_v2_simulation/schemas/comparison.py 100.00% <100.00%> (ø)
spp_api_v2_simulation/schemas/run.py 100.00% <100.00%> (ø)
spp_api_v2_simulation/schemas/scenario.py 100.00% <100.00%> (ø)
spp_api_v2_simulation/schemas/simulation.py 100.00% <100.00%> (ø)
... and 11 more

... and 13 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Copy link

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a new module, spp_api_v2_simulation, to provide a comprehensive REST API for managing and executing simulation scenarios. The changes are extensive, adding new models, services, routers, Pydantic schemas, and tests to support CRUD operations on scenarios, runs, and comparisons, as well as analytics and aggregation endpoints.

The overall structure is good, but there are a few significant areas for improvement. The most critical issue is the duplication of business logic between the newly introduced routers and a SimulationApiService that appears to be unused by the API endpoints. This should be refactored to have the routers delegate logic to the service layer, making the code more maintainable. Additionally, there is considerable duplication among the Pydantic schema definitions, which should be consolidated. I've also noted a security concern regarding potential information leakage in exception handling and a minor point about an empty ir.model.access.csv file.

Note: Security Review did not run due to the size of the PR.

_logger = logging.getLogger(__name__)


class SimulationApiService:

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

critical

There is a significant amount of duplicated logic between the API routers (e.g., in spp_api_v2_simulation/routers/scenario.py) and this service class. The routers currently implement business logic for creating, updating, and fetching simulation data directly, while this service contains a parallel, unused implementation of the same logic.

To improve maintainability and adhere to a clear separation of concerns, the routers should be refactored to be a thin layer that handles HTTP requests/responses and delegates all business logic to this service. This will create a single source of truth for the simulation operations and make the codebase cleaner and easier to manage.

@@ -0,0 +1,267 @@
# Part of OpenSPP. See LICENSE file for full copyright and licensing details.
"""Pydantic schemas for Simulation API."""

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

This file defines several Pydantic schemas (e.g., CreateScenarioRequest, ScenarioResponse, ComparisonResponse) that are duplicates or slight variations of schemas defined in other files within this directory (e.g., scenario.py, comparison.py). The API routers appear to use the schemas from the other, more specific files, which suggests that many of the schemas in this file are unused in the context of the API.

This duplication can lead to confusion and maintenance issues. It is highly recommended to consolidate the schemas, remove the duplicates, and ensure there is a single, authoritative definition for each schema. This file should likely only contain the schemas that are actually used from it, such as TemplateResponse and TemplateListResponse.

Comment on lines +134 to +139
except Exception as e:
_logger.exception("Failed to create comparison")
raise HTTPException(
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
detail=f"Failed to create comparison: {str(e)}",
) from None

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

In the generic except Exception as e: block, the exception details str(e) are included in the HTTP response's detail message. This can potentially leak sensitive internal information, such as database details or library internals, to the API client, which is a security risk.

The current implementation already logs the full exception with _logger.exception, which is excellent for debugging. The client-facing error message, however, should be generic to avoid leaking implementation details. This same issue is present in the exception handlers in run.py and scenario.py as well.

Suggested change
except Exception as e:
_logger.exception("Failed to create comparison")
raise HTTPException(
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
detail=f"Failed to create comparison: {str(e)}",
) from None
except Exception as e:
_logger.exception("Failed to create comparison")
raise HTTPException(
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
detail="Failed to create comparison",
) from e

@@ -0,0 +1 @@
id,name,model_id:id,group_id:id,perm_read,perm_write,perm_create,perm_unlink

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

This ir.model.access.csv file only contains a header row and is otherwise empty. While API access is correctly handled via scopes, an empty access file is unusual. If no new models require specific access rights for regular Odoo users, this file could be removed to avoid confusion. If it's intended as a placeholder for future rules, adding a comment to that effect would be helpful for other developers.

jeremi added 2 commits March 5, 2026 16:25
Prevent internal exception messages from leaking to API callers via
HTTPException detail strings. Full exception info is still logged via
_logger.exception for server-side diagnosis.
@jeremi jeremi force-pushed the feat/api-v2-simulation branch from e0859a4 to c36f076 Compare March 5, 2026 09:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant